<html>
<head><meta charset="utf-8"><title>Help with collecting stderr across dependencies · t-cargo · Zulip Chat Archive</title></head>
<h2>Stream: <a href="https://rust-lang.github.io/zulip_archive/stream/246057-t-cargo/index.html">t-cargo</a></h2>
<h3>Topic: <a href="https://rust-lang.github.io/zulip_archive/stream/246057-t-cargo/topic/Help.20with.20collecting.20stderr.20across.20dependencies.html">Help with collecting stderr across dependencies</a></h3>

<hr>

<base href="https://rust-lang.zulipchat.com">

<head><link href="https://rust-lang.github.io/zulip_archive/style.css" rel="stylesheet"></head>

<a name="220399774"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/246057-t-cargo/topic/Help%20with%20collecting%20stderr%20across%20dependencies/near/220399774" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/246057-t-cargo/topic/Help.20with.20collecting.20stderr.20across.20dependencies.html#220399774">(Dec 18 2020 at 17:08)</a>:</h4>
<p>I'm trying to work on <a href="https://github.com/rust-lang/cargo/issues/8749">https://github.com/rust-lang/cargo/issues/8749</a>. I found that most of this work happens in <a href="https://github.com/rust-lang/cargo/blob/27dba04f406a870f00241b53cbda74411cd53c9f/src/cargo/core/compiler/mod.rs#L1161-L1162">https://github.com/rust-lang/cargo/blob/27dba04f406a870f00241b53cbda74411cd53c9f/src/cargo/core/compiler/mod.rs#L1161-L1162</a>, which runs for every line of stderr. What would be a good approach to take for buffering this across crates? Is there some 'global' state that persists across all invocations of rustc?</p>



<a name="220400517"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/246057-t-cargo/topic/Help%20with%20collecting%20stderr%20across%20dependencies/near/220400517" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/246057-t-cargo/topic/Help.20with.20collecting.20stderr.20across.20dependencies.html#220400517">(Dec 18 2020 at 17:12)</a>:</h4>
<p>I did find <code>JobState</code> but I think that's only for a single <code>rustc</code> invocation</p>



<a name="220403026"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/246057-t-cargo/topic/Help%20with%20collecting%20stderr%20across%20dependencies/near/220403026" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/246057-t-cargo/topic/Help.20with.20collecting.20stderr.20across.20dependencies.html#220403026">(Dec 18 2020 at 17:31)</a>:</h4>
<p><span class="user-mention" data-user-id="239881">@Josh Triplett</span> let me know if you have time to help with this :)</p>



<a name="220403089"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/246057-t-cargo/topic/Help%20with%20collecting%20stderr%20across%20dependencies/near/220403089" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/246057-t-cargo/topic/Help.20with.20collecting.20stderr.20across.20dependencies.html#220403089">(Dec 18 2020 at 17:32)</a>:</h4>
<p>mainly I want to unblock <a href="https://github.com/rust-lang/rust/pull/77492">https://github.com/rust-lang/rust/pull/77492</a></p>



<a name="220456915"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/246057-t-cargo/topic/Help%20with%20collecting%20stderr%20across%20dependencies/near/220456915" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/246057-t-cargo/topic/Help.20with.20collecting.20stderr.20across.20dependencies.html#220456915">(Dec 19 2020 at 06:28)</a>:</h4>
<p>Are you looking to persist the lines or the count?</p>



<a name="220456918"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/246057-t-cargo/topic/Help%20with%20collecting%20stderr%20across%20dependencies/near/220456918" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/246057-t-cargo/topic/Help.20with.20collecting.20stderr.20across.20dependencies.html#220456918">(Dec 19 2020 at 06:28)</a>:</h4>
<p>For the lines, I would suggest capturing the output of each crate build into a file somewhere under the target directory, so that they can be "replayed" when needed.</p>



<a name="220456919"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/246057-t-cargo/topic/Help%20with%20collecting%20stderr%20across%20dependencies/near/220456919" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/246057-t-cargo/topic/Help.20with.20collecting.20stderr.20across.20dependencies.html#220456919">(Dec 19 2020 at 06:28)</a>:</h4>
<p>either - preferably the lines but that sounds harder than just preserving the count, so happy to start with the count instead</p>



<a name="220456926"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/246057-t-cargo/topic/Help%20with%20collecting%20stderr%20across%20dependencies/near/220456926" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/246057-t-cargo/topic/Help.20with.20collecting.20stderr.20across.20dependencies.html#220456926">(Dec 19 2020 at 06:29)</a>:</h4>
<p>Honestly, I think the lines might be easier, unless there's an obvious place to keep global state.</p>



<a name="220456927"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/246057-t-cargo/topic/Help%20with%20collecting%20stderr%20across%20dependencies/near/220456927" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/246057-t-cargo/topic/Help.20with.20collecting.20stderr.20across.20dependencies.html#220456927">(Dec 19 2020 at 06:29)</a>:</h4>
<p>Cargo already has some logic for persisting build output and replaying it later.</p>



<a name="220456928"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/246057-t-cargo/topic/Help%20with%20collecting%20stderr%20across%20dependencies/near/220456928" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/246057-t-cargo/topic/Help.20with.20collecting.20stderr.20across.20dependencies.html#220456928">(Dec 19 2020 at 06:29)</a>:</h4>
<p>I think you'd just need to make that work for individual crates.</p>



<a name="220456929"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/246057-t-cargo/topic/Help%20with%20collecting%20stderr%20across%20dependencies/near/220456929" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/246057-t-cargo/topic/Help.20with.20collecting.20stderr.20across.20dependencies.html#220456929">(Dec 19 2020 at 06:29)</a>:</h4>
<p><span class="user-mention silent" data-user-id="239881">Josh Triplett</span> <a href="#narrow/stream/246057-t-cargo/topic/Help.20with.20collecting.20stderr.20across.20dependencies/near/220456927">said</a>:</p>
<blockquote>
<p>Cargo already has some logic for persisting build output and replaying it later.</p>
</blockquote>
<p>oh perfect! where can I find that?</p>



<a name="220457084"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/246057-t-cargo/topic/Help%20with%20collecting%20stderr%20across%20dependencies/near/220457084" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/246057-t-cargo/topic/Help.20with.20collecting.20stderr.20across.20dependencies.html#220457084">(Dec 19 2020 at 06:34)</a>:</h4>
<p>/me tries to remember where that lives...</p>



<a name="220457102"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/246057-t-cargo/topic/Help%20with%20collecting%20stderr%20across%20dependencies/near/220457102" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/246057-t-cargo/topic/Help.20with.20collecting.20stderr.20across.20dependencies.html#220457102">(Dec 19 2020 at 06:35)</a>:</h4>
<p>I remember that there was some careful caching logic to make sure it was safe to just replay the output...</p>



<a name="220457144"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/246057-t-cargo/topic/Help%20with%20collecting%20stderr%20across%20dependencies/near/220457144" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/246057-t-cargo/topic/Help.20with.20collecting.20stderr.20across.20dependencies.html#220457144">(Dec 19 2020 at 06:36)</a>:</h4>
<p>(I remember partly because there were previously bugs in that logic...)</p>



<a name="220457202"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/246057-t-cargo/topic/Help%20with%20collecting%20stderr%20across%20dependencies/near/220457202" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Josh Triplett <a href="https://rust-lang.github.io/zulip_archive/stream/246057-t-cargo/topic/Help.20with.20collecting.20stderr.20across.20dependencies.html#220457202">(Dec 19 2020 at 06:38)</a>:</h4>
<p>I honestly don't know.</p>



<a name="220457538"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/246057-t-cargo/topic/Help%20with%20collecting%20stderr%20across%20dependencies/near/220457538" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Joshua Nelson <a href="https://rust-lang.github.io/zulip_archive/stream/246057-t-cargo/topic/Help.20with.20collecting.20stderr.20across.20dependencies.html#220457538">(Dec 19 2020 at 06:50)</a>:</h4>
<p>no problem, I'll find it</p>



<hr><p>Last updated: Aug 07 2021 at 22:04 UTC</p>
</html>